18. Monitoring

Which of the following DAGs might benefit from an SLA?
SOLUTION:
  • A DAG which determines how to allocate marketing money next hour based on the last hour’s performance
  • A DAG which generates analytics that the executive team receives via email every morning

System Metric

Which of the following is an example of an Airflow system-level metric?
SOLUTION: Task failures for all DAGs in the system

Missed SLAs

Which of the following components might need to be scaled up if your SLAs are routinely missed?

SOLUTION: Workers

Email

Which of the following can Airflow be configured to send on DAG or task state change?
SOLUTION: Emails

What state changes can Airflow send notifications for?
SOLUTION:
  • Task Success, Failure, and Retry
  • DAG Success, Failure, and Retry

Notifications

True or False: Notifications can be sent individually for both entire DAGs or tasks.
SOLUTION: True

Next Concept